Risk Management in EVA 
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Agenda 


Topic: Use of Risk Management in EVA 

Background 

Approach 

Usefulness/Conclusion 



EVA Office at NASA - JSC 


• EVA Office at JSC provides 

- Space Suits used for EVA’s 

• Life support system 

• Soft-goods (bladder, MMOD 
and thermal protection) 

• Communication, and even 
propulsion 

- Tools & crew aids used for EVA’s 

• Hand-tools 

• Transition and positioning 
aids 

- Handrails 

- Portable foot restraints 

- Tool caddies 

- Mission Support 

• Crew training (ground based 
eg NBL, VRL) 

• On-orbit execution (hardware 
supply, preparation, anomaly 
resolution 


• EVA is an organization with a product 
focus, a strong customer orientation 
(ISSP, SSP, Cx), with significant 
integration activities 



EVA Project Risk Management: 

Why and When 


Why? 

• Agency interest in adopting “best 
practices” resulted in ISSP accepting 
challenge to implement RM 

• ISSP is primary customer, and 
mandated requirement to EVA (EVA 
as vendor/service supplier) 

When? 

• Informally, RM occurred since 
inception with use of “Threat Lists” 

• The formal, structured system 
(database and graphics) began for 
EVA at “mid-life” 


Comments 

• Continuous improvement initiative 


• EVA’s strong customer orientation 


Note: can implement formal RM approach 
long after developing POS and the 
Project Definition Statement 






EVA Office Risk Management: How 


Systematic approach for ISSP & EVA 

• Risk Matrix (5 X 5) - see illustration 

- Likelihood 

- Consequence 

• Brainstormed initial listing with 
preliminary Risk Scores 

• Established database to capture and 
track risk status and efforts to mitigate. 
Status reporting (monthly) 

- Summary Matrix of Risks 

- Narrative updates 

• Risk Manager established for ISSP to 
manage Risk System 

- Needed method to elevate issues 


Comments 

• Simple estimations for values assigned 
to probabilities and impacts (Ordinal 
versus Cardinal) 

• Used Risk Initiation Info Sheet, 
assigned ID number, rated each for 
probability & impacts, identified 
mitigation plans (Contingencies 
developed later) 


• Risk Tracking System included feature 
to allow “promoting” and “demoting” 
risks by EVA Office. This facilitated 
accepting, transferring, mitigating, and 
closing risks. 
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EVA Office Risk Guide Card - Example 


What is the Likelihood the Risk Will Happen? 

Level 

Chance of Occurrence 

Your Typical Approach and Processes 

a 

Not Likely: 

Less than 10% chance 

. . . Will effectively avoid this risk based on standard practices 

b 

Low Likelihood: 

1 1% - 25% chance 

. . .Have usually avoided this type of risk with minimal 
oversight in similar cases 

c 

Moderate: 
26% - 50% chance 

. . .May avoid this risk, but workarounds will be required 

d 

Highly Likely: 
51% - 75% chance 

. . .Cannot avoid this risk with standard practices, but a different 
approach may work 

e 

Near Certainty: 
Greater than 76% chance 

. . .Cannot avoid this risk with standard practices, may not be 
able to mitigate 



Given the risk is realized, what would be the magnitude of the impact? 


Level 

Technical Performance 

Schedule Impact 

Cost 

(millions) 




Minimal schedule slip but able to 


0> 

o 

1 

Minimal or No technical Impact, same 

meet need dates w/o add’l resources. 

Cost increase 

Almost Negligible 

approach retained. 

Critical path unaffected 

<0.5 

S3 


Minor technical shortfall and/or small 

Minimal schedule slip requiring add’l 


QJ 

2 

reduction in margin. Minor changes 

resources to meet need dates. Critical 

Cost increase 

3 

Marginal 

may be needed. 

path unaffected. 

0.5 -1.0 

Vl 

S3 


Moderate technical shortfall and/or 

Significant schedule slip. 


3 

Moderate 

significant reduction in margin. 
Workarounds available. 


Cost increase 
1.0-5 

o 

tj 

4 

No remaining margin. Severe technical 
shortfall. 

Major schedule slip. Will miss 
Milestone date. 

Cost increase 

Critical 



5-10 


5 

Catastrophic 

Unacceptable, will result in technical 
performance failure w/ no known 
workarounds. Ship will be 
undeliverable. 

Major (critical) schedule slip. 

Cost increase 
>10 



ANALYSIS GUIDE 




HIGH - Major program disruption, immediate priority 
management action required. 

MODERATE - Moderate disruption, possible 
management action required 
LOW - Minimum impact 


Source: EVA RM Plan (draft) 
circa 2004 
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EVA Office Risk Initiation Information Sheet 







Criteria for Closing a 

1 . Risk has been fully mitigated or is no longer present 

- All mitigation tasks have been met or accomplished 

2. Risk is mitigated down to the “Green” 2x2 or lower area and it is 

determined sufficient mitigation has been accomplished and no further 
mitigation is necessary, desired or planned 

- Will not need to revisit issue in the future (subjective) 

Process for Closing a Risk 

- Notification (not approval) at PRAB for Non-TPR Risks 

- Approval at PRAB for TPR Risks 

- Document closure rationale with sufficient data (technical, cost, 
schedule) to support the recommendation 

• Already part of IRMA software 




Source: ISSP 
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Criteria for Accepting a 

Risk 

1 . It is not feasible or desirable to fully mitigate a Risk or to mitigate the Risk 

any further 

- May not be possible, technically practical, or not cost effective 

• i.e. Reliance on Russians to provide attitude control & propellant 
resupply (Develop Propulsion Module) 

- May exceed time (schedule), money, or personnel to fully mitigate 

2. Risk is in the “Red”, “Yellow”, or above a “Green” 2x2 area 

- Or could be any type of Risk or Watch Item where we desire visibility 
of “acceptance of risk” (subjective) 


Process for Accepting a Risk 

- Approval at PRAB by Program Manager 

- Document closure [sic] rationale with sufficient data (technical, cost, 
schedule) to support the recommendation 

- Re-assess all accepted risk periodically 

• Minimum twice per year 


K 

E 

L 

I 

H 

O 

O 

D 









1 


1 l 


1 















CONSEQUENCE 


Source: ISSP 
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ISS IRMA Reference Card 
- Data Entry Requirements 


Mandatory Field 

Concern 

Cost Issue 

Watch Item 

Risk 

Title 

7 

7 

7 

7 

Description 



*/ 

xi 

ECD 



7 

7 

Most Likely Mit. Cost 



7 

7 

High & Low Mit. Cost 



if 

if 

7 

Mit. Budget Amount 


7 

7 

7 

Cost Category 


7 

7 

7 

Cost Breakdown 




7 

Likelihood Score 

7 

7 

7 

if 

7 

Consequence Score 

7 

V 


7 

Impact/Consequence 



yl 

J 

Closure/Acceptance Criteria 



W 

V 

s/ 

Flights/Stages Affected 

7 

7 

7 

7 

Orgs Affected 





Status 





x/ 







Mitigation Plan Overview 

Kev 




7 

Mitigation Tasks 

V = New 




7 

Source: ISSP 

V = Proposed 

nr 


XfiJ EVA Office Risk Activity Summary 


Type of Risk 


Closed Opened 


Open Now 


Risk 

Watch Item 
Cost Issue 
Concern 
Total 


0 

1 

2 

0 

3 


0 

2 

1 

1 

4 


0 

X 

Y 

Z 

(0+X+Y+Z) 


Watch Item (TPR) Transferred this period : 

50XX “Lack of (insert phrase here)” 

• Transferred from EVA to Vehicle Office 

• Authority and resources necessary to handle risk are Vehicle 
Office 


Source: J. Hall 
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EVA Significant Change Summary 


Transfers 

- To: OB 50XX Watch Item Lack of(insert phrase here) 
Openings 


- 51 XX Watch Item Operations (insert phrase here) 

- 51XX Watch Item Possible Need for Additional Hardware 

- 51XX Cost Issue EVA Operations 

- 51XX Concern EVA hardware compatibility with next generation computers 
Closures 


Num. 

Risk Type 

Risk Title 

Closure Rationale 

50xx 

Watch Item 

Completion of Budget Transfer 

Received transfer of Budget from HQ 

48xx 

Cost Issue 

EVA Materials 

To be covered within budget. 

50xx 

Cost Issue 

Lack of (insert phrase here) 

Funding approved via CCB 


Conversions 

- 47xx Cost Issue to Watch Item New Design (insert phrase here) 

Pending 

- 46xx Watch Item EVA (insert phrase here) - cost impacts of are under review 


Source: J. Hall 
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Integrated Risk Management Application 
(XA) Matrix, March 31, 2004 


Corrective/Preventative Actions 


Watch Items 


■ 50xx - EVA hardware inventory 

■ 49xx - Reduced capability 

■ 49xx - EMU Battery logistics 

■ 48xx - Ops constraints 

■ 47xx - New Design Hardware Development 

■ 45xx - Tool Torque 

■ 45xx - EVA Limit Loads 

■ 46xx - EVA hardware inventory 


Cost Issues 


■ 50xx - EMU inventory 

■ 51 xx - EVA Operations 

■ 48xx - Hardware Replacement 

■ 48xx - New Requirement design 


Continual Improvement 


K 

E 

L 

I 

H 

O 

O 

D 


Program Risk Matrix for the EVA 
Projects Office 


Low 


C — Cost 


3 4 5 

CONSEQUENCE 


Medium 


S-Schedule 


High 


T-Technical 


- Top Program Risk (TPR) 


A -Proposed Top Program Risk (TPR) 


I - Top Organization Risk (TOR) 


□ - Proposed Top Organization Risk (TOR) 


Risks (L x C) 


No Risks Elevated 


Risks Being Watched/Monitored 

A Watch: 50xx - Lack (insert phrase here) 

Transferred to Vehicle Office in last period 

Watch: 44xx - Maintenance (insert phrase here) 
Watch: 45xx - Tool Loads 


During March 2004 
EVA had no Top 
Program Risks, so the 
Matrix is empty. 


Source: J. Hall 
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Integrated Risk Management Application (IRMA) Home List of OPEN XA Risks 

Status as of March 17, 2004 




IRMA 









Flights 

Orgs 

Risk 


MO 

Type 

Num. 

Owner 

Title 

FY04 

FY05 

FY06 

FY07 

FY08 

FY09 

Affected 

Affected 

Level 

LxC 

XA 

Cost Issue 

40XX 

jdoe 

ABCD facility 

0 

0.7 

0.668 

0.689 

0.711 

0.734 


XA 


0x0 

XA 

Cost Issue 

40XX 

jdoe 

ABCDE hardware 

0 

0.6 

0.64 

0.68 

0.73 

0 


XA 


3x3 

XA 

Cost Issue 

40XX 

jdoe 

BCDE hardware 

0 

0.2 

0.4 

2.3 

4 

1.6 


XA 


3x3 

XA 

Cost Issue 

40XX 

jdoe 

BCDEF facility 

0 

0.3 

2.5 

2.5 

0 

0 


XA 


0x0 

XA 

Watch Item 

40XX 

jdoe 

CDEF hardware 

0 

0 

0 

0 

0 

0 


XA 

TOR 

3x3 

XA 

Watch Item 

40XX 

jdoe 

CDEFG facility 

0 

1 

1 

1 

1 

0 


XA 


2x3 

XA 

Watch Item 

40XX 

jdoe 

DEFG facility 

0 

0 

0 

0 

0 

0 


XA 

TOR 

2x3 

XA 

Watch Item 

40XX 

jdoe 

DEFGH hardware 

0 

0 

0 

0 

0 

0 


XA 

TOR 

4x3 

XA 

Concern 

40XX 

jdoe 

ABCD facility 

0 

0 

0 

0 

0 

0 


XA 


3x4 

XA 

Watch Item 

40XX 

jdoe 

ABCDE hardware 

0 

4.2 

0 

0 

0 

0 


XA, SA 


2x3 

XA 

Concern 

40XX 

jdoe 

BCDE hardware 

0 

0 

0 

0 

0 

0 


XA 


3x3 

XA 

Concern 

40XX 

jdoe 

BCDEF facility 

0 

0 

0 

0 

0 

0 


XA 


3x4 

XA 

Cost Issue 

40XX 

jdoe 

CDEF hardware 

0 

5 

7 

7 

7 

7 


XA 


1 x 1 

XA 

Cost Issue 

40XX 

jdoe 

CDEFG facility 

0 

0 

0.232 

0.239 

0.246 

0.253 


XA 


0x0 

XA 

Cost Issue 

40XX 

jdoe 

DEFG facility 

0 

1 

1 

1 

1 

1 


XA 


1 x 1 

XA 

Watch Item 

40XX 

jdoe 

DEFGH hardware 

0 

1.5 

2 

1 

0 

0 


XA 

TOR 

4x3 

XA 

Cost Issue 

40XX 

jdoe 

ABCD facility 

0 

1.5 

1.5 

1.5 

0 

0 


XA 


1 x 2 

XA 

Cost Issue 

40XX 

jdoe 

ABCDE hardware 

0 

8.6 

6.4 

1.6 

0.37 

0 


XA 

TOR 

2x3 

XA 

Cost Issue 

40XX 

jdoe 

BCDE hardware 

0.95 

1.2 

0.7 

0 

0 

0 


XA 

TOR 

1 x 3 

XA 

Cost Issue 

40XX 

jdoe 

BCDEF facility 

0 

0.372 

0.384 

0.268 

0.278 

0.288 


XA 


1 x 3 

XA 

Cost Issue 

40XX 

jdoe 

CDEF hardware 

0 

0.525 

0 

0 

0 

0 


XA 


1 x 3 

XA 

Cost Issue 

40XX 

jdoe 

CDEFG facility 

0 

0.65 

0.794 

0.942 

1.095 

1.252 


XA 


1 x 2 

XA 

Cost Issue 

40XX 

jdoe 

DEFG facility 

0 

1.1 

1.2 

1.3 

0.6 

0 


XA 


1 x 2 

XA 

Watch Item 

40XX 

jdoe 

DEFGH hardware 

0 

0 

0 

0 

0 

0 


XA 

TOR 

4x3 

XA 

Cost Issue 

40XX 

jdoe 

ABCD facility 

0 

0 

0 

0 

0 

6.6 


XA 


3x4 

XA 

Watch Item 

40XX 

jdoe 

ABCDE hardware 

0 

0 

0 

0 

0 

0 

ULF1 , 8S, 
14P, 15P, 9S, 
16P 

XA, OE 

TOR 

4x3 

XA 

Watch Item 

40XX 

jdoe 

BCDE hardware 

0 

0 

0 

0 

0 

0 

ULF1 , 8S, 
14P, 15P, 9S, 
16P 

XA, OE 

TOR 

2x2 

XA 

Concern 

40XX 

jdoe 

BCDEF facility 

0 

0 

0 

0 

0 

0 


XA 


1 x 1 

XA 

Cost Issue 

40XX 

jdoe 

CDEF hardware 

0.1 

0.001 

0 

0 

0 

0 



TOR 

1 x 5 

XA 

Watch Item 

40XX 

jdoe 

CDEFG facility 

0 

0.6 

0 

0 

0 

0 


XA 

TOR 

4x3 

XA 

Cost Issue 

40XX 

jdoe 

DEFG facility 

0 

0.6 

0.64 

0.68 

0.73 

0 


XA 


3x3 

XA 

Cost Issue 

40XX 

jdoe 

DEFGH hardware 

0.7 

0.6 

0 

0 

0 

0 


XA 

TOR 

3x3 

XA 

Watch Item 

40XX 

jdoe 

ABCD facility 

0.0001 

0 

0 

0 

0 

0 


XA 

TOR 

2x4 

XA 

Watch Item 

40XX 

jdoe 

ABCDE hardware 

-0.6 

0 

0 

0 

0 

0 

LF1 , ULF1.1, 

XA, OC, OE, 

TPR 

5x3 












12A, 12A.1 , 

CA, DA, EA, 














13A, 13A.1 , 
15A 

SA, MA, OB 



XA 

Watch Item 

40XX 

jdoe 

BCDE hardware 

4 

2.4 

0 

0 

0 

0 



TOR 

3x3 

XA 

Concern 

40XX 

jdoe 

BCDEF facility 

0 

0 

0 

0 

0 

0 




1 x 1 
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Source: J. Hall 



ISS Watch Item: 50XX Summary Report 

Open Date: 1/27/2004 Status as of 3/25/2004 ECD: 3/6/2005 

TRANSFERRED to ISS Vehicle Office 

Title: Lack of (insert phrase here) 

Description: Currently, there is no certified capability other than 

Risk Owner: Doe, John 

Likelihood: 5 Consequence: 2(C), 2(T) 

Impact/Consequence: Without 

Organization: XX MO Affected: CA, DA, EA, MA, OB,SA, XA 

Flights Affected: 12A, 12A.1, 13A, 13A.1, 15A, LF-1, ULF-1.1, 

Total Most Like. Mit. Cost ($M): 0.15 Total Budget ($M): 0.75 Cost of Inaction ($M): 10 

Current Status: 

See Risk Owner 


Source: J. Hall 
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EVA Project RM Usefulness 


• Identifying risks improves awareness of vulnerabilities and forces 
consideration of mitigation and contingencies 

- Use of different types 

• Allows flexibility to elevate as appropriate 

• Reduces the number of sudden surprises by identifying “emerging 
risks” 

• Tracking risks on monthly basis keeps project engineers and management 
focused on choices (accept/reject, mitigate, transfer) 

• Although initially unpopular with engineering workforce (additional burden 
with “unknown benefit”), 2 years later it is part of the culture and an 
expected metric 

• Risk System now “feeds” the annual budget development cycle for ISSP 
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